perf: mirror ASOF equality-key filters to right - #25574

Codecov Report

❌ Patch coverage is 74.07407% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.48%. Comparing base (92559af) to head (c7522fa).
⚠️ Report is 1 commits behind head on main.

@@            Coverage Diff             @@
##             main   #25574      +/-   ##
==========================================
- Coverage   82.48%   82.48%   -0.01%     
==========================================
  Files        1140     1140              
  Lines      438540   438520      -20     
  Branches   438540   438520      -20     
==========================================
- Hits       361751   361722      -29     
- Misses      54946    54954       +8     
- Partials    21843    21844       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Hi @jayzhan211 and @2010YOUY01, this follow-up is now restacked and ready for review. Would you mind taking a look when you have time? Thanks!

Thanks @Xuanwo , one suggestion here

The hand-written col = literal match covers only one shape and duplicates infer_join_predicates_impl in this file. Every matching pair has equal keys, so any deterministic predicate over only left key columns also holds for the matching right keys. Reusing the helper also mirrors IN, ranges and OR on the key. The same-type check can go: coercion already wraps mismatched keys in CAST, so they are never direct columns. I tried this locally and asof_join.slt passes unchanged. It's fine to do this in a follow-up.

-                // A literal comparison on an equal, same-typed key has the
-                // same value for every matching pair. Mirroring it to the
-                // right can prune groups without changing the ASOF candidate.
-                let mut right_predicates = Vec::new();
-                for predicate in &push_predicates {
-                    ...
-                }
+                // Every matching pair has equal key values, so a deterministic
+                // predicate over left keys holds for the matching right keys.
+                let join_col_keys = join
+                    .on
+                    .iter()
+                    .filter_map(|(l, r)| Some((l.try_as_col()?, r.try_as_col()?)))
+                    .collect::<Vec<_>>();
+                let mut inferred = InferredPredicates::new(JoinType::Inner);
+                infer_join_predicates_impl::<true, false>(
+                    &join_col_keys,
+                    &push_predicates,
+                    &mut inferred,
+                )?;
+                let right_predicates = inferred
+                    .predicates
+                    .into_iter()
+                    .filter(|p| {
+                        p.column_refs()
+                            .iter()
+                            .all(|c| join.right.schema().is_column_from_schema(c))
+                    })
+                    .collect::<Vec<_>>();

With this change, both of these prune the right input as well:

SELECT * FROM l ASOF JOIN r MATCH_CONDITION (l.ts >= r.ts) ON l.k = r.k WHERE l.k IN (1, 3);
SELECT * FROM l ASOF JOIN r MATCH_CONDITION (l.ts >= r.ts) ON l.k = r.k WHERE l.k > 1;

The hand-written col = literal match covers only one shape and duplicates infer_join_predicates_impl in this file. Every matching pair has equal keys, so any deterministic predicate over only left key columns also holds for the matching right keys. Reusing the helper also mirrors IN, ranges and OR on the key. The same-type check can go: coercion already wraps mismatched keys in CAST, so they are never direct columns. I tried this locally and asof_join.slt passes unchanged. It's fine to do this in a follow-up.

I think this util is a neat idea, but a bit hard to wrap my head around and get convinced it's safe to reuse; However we can implement something similar for ASOF joins.

Performance

Q09 has 1M rows on each side across 10K equality groups. WHERE l.key = 42 retains one group and produces 100 rows with both revisions. On an Apple M4 Max, using release builds and two 7-iteration runs per revision, the median elapsed time across all 14 iterations was:

Revision Median elapsed
main without this optimization (b172f73f0) 21.5 ms
This change (b3fd5f1f9) 2.8 ms
This selective keyed workload is about 7.8x faster (87% lower elapsed time). The benchmark file and command were the same for both binaries:

target/release/benchmark_runner asof_join --query 9 --iterations 7

The branch was subsequently synced through 21a3215b6; those upstream-only commits do not touch the optimizer or ASOF paths exercised by Q09.

nit: We could scale this benchmark query up so a run takes around 1s. At ~2ms per run, the numbers are easily swayed by noise, e.g. a false-positive 2x slowdown.

Thanks @jayzhan211 and @2010YOUY01. I kept this PR focused and opened #25728 with an ASOF-specific key mapping.

I also scaled Q09 to about 1s on the baseline and reran it. The new median is 972.1ms vs 78.5ms across 14 runs.

Thanks @Xuanwo and @2010YOUY01 !